Video signal processing

ABSTRACT

In video processing, a motion compensated process and an equivalent linear process are performed on an input signal, the processing errors of which are computed and compared. The comparison is then used to control a mix between the outputs of the two processes to produce an output signal.

[0001] This invention relates to video processing and particularly tomotion compensation of video processes.

[0002] It is a well-known technique in video processing, to identify amotion vector for each pixel and to shift pixels in accordance withthose vectors. Such motion compensation is of benefit in myriad videoprocesses, of which standards conversion is a good example. A motioncompensated process will be expected to perform considerably better thanthe equivalent linear process, although at a substantial extra cost interms of hardware complexity or software processing requirement.

[0003] It is further known in simpler arrangements to switch between twoprocesses, one spatial and one temporal, on the basis of whether motionis detected. This has the advantage of avoiding motion smear byswitching to spatial processing, but typically suffers from adaptionartefacts.

[0004] It is an object of one aspect of the present invention to providea method of taking motion into account, which is less complex andinvolves less processing than full motion compensation, but whichnonetheless offers significant improvements over the equivalent linearprocess.

[0005] There is disclosed in EP 0 702 880 a method of reducing thedeleterious effects of motion in a video signal process, comprising thesteps of conducting the video signal process on a video signal at highbandwidth without motion compensation; conducting a like video signalprocess without motion compensation at low bandwidth to produce a lowbandwidth control signal; conducting a similar video signal process withmotion compensation at low bandwidth to produce a low bandwidth motioncompensated signal; comparing the control and motion compensated signalsto produce an error signal and subtracting the error signal in the highbandwidth video signal process.

[0006] The present invention consists in one aspect in a method of videoprocessing comprising the steps of receiving an input signal; conductinga motion compensated video process on the input signal to form a motioncompensated signal; conducting an equivalent linear process on the inputsignal to form a linear signal; forming a measure of the relative errorsin the motion compensated process and the linear process respectively;and utilising said measure to form an output signal which is a mixbetween the motion compensated signal and the linear signal, therelative proportions of the motion compensated signal and linear signalin said mix being determined by said measure.

[0007] The invention will now be described by way of example withreference to the accompanying drawings, in which:

[0008]FIG. 1 is a block diagram of apparatus according to an embodimentof the present invention;

[0009]FIG. 2 is a block diagram of apparatus according to anotherembodiment of the present invention;

[0010]FIG. 3 is a block diagram of apparatus according to a furtherembodiment of the present invention; and

[0011]FIG. 4 is a block diagram of apparatus according to a stillfurther embodiment of the present invention.

[0012] Referring initially to FIG. 1, in one embodiment, the systemgenerates two standards converted images (one linear conversion (102)and one motion compensated (104)) from an input signal (100). Anadaption signal is then generated (106) by comparison of the converteroutputs, and is used to control the mixing (108) of the two signals toproduce the output signal (110).

[0013] In this manner, the two different conversion signals arecompared, in order to determine which is more appropriate for thematerial currently being processed. The comparison controls the mix, sothat, for example, at instances where motion compensated standardsconversion gives better results, the signal from block 104 is preferablyused as a higher proportion of the mix. The system thus produces theoptimum output from a mix of the two different converters, rather thansimply switching from one to the other.

[0014] Reference is directed to EP 0 702 880 for more detail of a methodof generating two standards converted images, one linear conversion andone motion compensated.

[0015] In another embodiment, shown in FIG. 2, the video input (100) ispassed to the linear standards converter (102) and to the motioncompensated standards converter (104). The output of the linearconversion (200) is then passed to a delay (202). The output (200) isthen subtracted from the delayed signal at block (204), producing adifference or error signal between the current picture and the previouspicture. A similar difference signal is calculated from the output ofthe motion compensated converter (206), through delay (208) andsubtraction (210).

[0016] The two difference signals are then used in generator (212) toderive an adaption signal indicating the relative merits of therespective converter outputs for the current picture material. The twoconverter outputs (200 and 206) are then mixed (214) in the proportionssuggested by the adaption signal to produce the output signal (110).

[0017] In a further embodiment, illustrated in FIG. 3, the input (100)is again passed to converters (102) and (104). At this point, acomparison is made between the fields currently being converted (asopposed to those currently being output by the converter at 200). Thustwo fields (300, 302), typically the current field and the fieldimmediately preceding it, are taken from the linear converter (102) andan error signal is calculated at 304. This error gives a measure of howsimilar the fields are. Similarly, an error is calculated at 310 betweentwo fields (306, 308) from the motion compensated converter (104). Thetwo error signals (312, 314) are then compared at 316 in order togenerate the adaption signal to control the mixer (318).

[0018] Thus in this embodiment, it is the comparison of the errorsignals which controls the mix. If the error between the fieldscurrently being converted linearly is larger than those being convertedwith motion compensation, it will be preferable to use the motioncompensated material to produce the output pictures (110). Similarly, ifthe error between the motion compensated fields is larger, the linearoutput is preferred.

[0019]FIG. 4 shows a standards converter according to a furtherembodiment of the invention. An input video signal (401) is input to astore and vertical interpolation block (402) and a motion estimator(403). The block (402) outputs four vertically interpolated fields whichare derived from the four input fields which straddle the temporalposition of the required output field. Each of these output fields isderived by intra-field vertical interpolation of the corresponding inputfield, has the required number of lines for the intended outputstandard, and is vertically aligned with the required output lineposition.

[0020] These four fields are fed to respective coefficient multipliers(404) to (407) where they are weighted in known manner according totemporal interpolation coefficients from a coefficient generator (408)and summed (409) so as to generate a linearly standards converted signal(410).

[0021] The four fields from the block (402) are also fed to respectivepicture shifters (411) to (414) where they are motion compensated inknown manner to the temporal position of the required output field bymotion vectors from the motion estimator (403). The shifted fields arethen weighted by the multipliers (415) to (418) and combined in theadder (419) to produce motion-compensated, converted signal (420).

[0022] The two central fields of the linear converter are compared inthe comparator (421) to produce a “linear conversion error signal”(422); and the two central motion compensated fields are compared in thecomparator (423) to produce a “motion compensated converter errorsignal” (424).

[0023] The error signals (422) and (424) are compared with each other ina comparator (425) which produces an adaptation control signal tocontrol the mixer (426), which derives the output signal (427) bycrossfading between the linearly converted signal (410) and themotion-compensated signal (420). The control is such as to favour thelinear signal when the error signal (424) is large with respect to theerror signal (422) and vice versa.

[0024] In the above description the same four vertically-convertedsignals from the block (402) are used for the linear conversion, themotion-compensated conversion and for the generation of the errorsignals. However, alternative embodiments advantageously use differentvertical interpolation characteristics (i.e. different coefficientsand/or different numbers of lines) for these three different functions.For example, the signals feeding the comparators could be optimised forthe control of the adaptation process, and the signals feeding theconverters could be optimised to reduce the visibility of conversionartefacts.

[0025] It should be noted that though in the example illustrated, theerror signals are calculated using only two fields, in other embodimentsany number of fields may be used (for example, four, from a four-fieldstandards converter).

[0026] It should be understood that this invention has been described byway of examples only and that numerous modifications are possiblewithout departing from the scope of the invention. For example, there isno restriction on the motion compensated method used.

[0027] This technique can be applied to standards conversion but wouldbe equally applicable in other areas where motion detection is useful.These include prediction based compression systems, interpolators andnoise reducers.

[0028] The techniques disclosed and claimed in this patent applicationcan with advantage be combined with any of the techniques disclosed andclaimed in the co-pending patent application PCT/GB01/01328 in the nameof the same applicant.

1. A method of video processing comprising the steps of receiving aninput signal; conducting a motion compensated video process on the inputsignal to form a motion compensated signal; conducting an equivalentlinear process on the input signal to form a linear signal; forming ameasure of the relative errors in the motion compensated process and thelinear process respectively; and utilising said measure to form anoutput signal which is a mix between the motion compensated signal andthe linear signal, the relative proportions of the motion compensatedsignal and linear signal in said mix being determined by said measure.2. A method according to claim 1, wherein said measure of relativeerrors is formed by forming a first error signal from the output of themotion compensated process and by forming a second error signal from theoutput of the linear process.
 3. A method according to claim 2, whereinthe first error signal is formed by forming a first difference signalbetween pictures in the motion compensated process, and the second errorsignal is formed by forming a second difference signal between picturesin the linear process.
 4. A method according to claim 2 or claim 3,wherein said measure is formed by comparing the first and second errorsignals. 5 A method of video processing comprising the steps ofconducting a motion compensated process on an input signal; conductingan equivalent linear process on the input signal; forming a first errorsignal from the output of the motion compensated process, forming asecond error signal from the output of the linear process; and utilisingthe error signals to form a mix between the output of the motioncompensated process and the output of the linear process.
 6. A method ofvideo processing comprising the steps of conducting a motion compensatedprocess on an input signal; conducting an equivalent linear process onthe input signal; forming a first difference signal from picturesundergoing the motion compensated process, forming a second differencesignal from pictures undergoing the linear process; comparing the firstand second difference signals; and utilising the comparison to form amix between the output of the motion compensated process and the outputof the linear process
 7. A method according to any one of the precedingclaims, wherein the motion compensation process comprises phasecorrelation.